Secondary device detection using a synchronous interface

ABSTRACT

A chiplet system can include a Serial Peripheral Interface (SPI) bus for communication. A controller or primary device coupled to the SPI bus can generate a message with read or write instructions for one or more secondary devices. In an example, the primary device can be configured to use information on a data input port or data input bus to determine a communication status of one or multiple secondary devices on the bus.

TECHNICAL FIELD

Embodiments of the disclosure relate generally to chiplet-basedelectronic systems and to communications in such systems.

BACKGROUND

Chiplets are an emerging technique for integrating various processingfunctionality. Generally, a chiplet system is made up of discrete chips(e.g., integrated circuits (ICs) on different substrate or die) that areintegrated on an interposer and packaged together. This arrangement isdistinct from single chips (e.g., ICs) that contain distinct deviceblocks (e.g., intellectual property (IP) blocks) on one substrate (e.g.,single die), such as a system-on-a-chip (SoC), or discretely packageddevices integrated on a board. In general, chiplets provide betterperformance (e.g., lower power consumption, reduced latency, etc.) thandiscretely packaged devices, and chiplets provide greater productionbenefits than single die chips. These production benefits can includehigher yields or reduced development costs and time.

Chiplet systems are generally made up of one or more applicationchiplets and support chiplets. Here, the distinction between applicationand support chiplets is simply a reference to the likely designscenarios for the chiplet system. Thus, for example, a synthetic visionchiplet system can include an application chiplet to produce thesynthetic vision output along with support chiplets, such as a memorycontroller chiplet, sensor interface chiplet, or communication chiplet.In a typical use case, the synthetic vision designer can design theapplication chiplet and source the support chiplets from other parties.Thus, the design expenditure (e.g., in terms of time or complexity) isreduced by avoiding the design and production of functionality embodiedin the support chiplets. Chiplets also support the tight integration ofIP blocks that can otherwise be difficult, such as those using differentfeature sizes. Thus, for example, devices designed during a previousfabrication generation with larger feature sizes, or those devices inwhich the feature size is optimized for power, speed, or heatgeneration—as can happen with sensors—can be more easily integrated withdevices having different feature sizes. Additionally, by reducing theoverall size of the die, the yield for chiplets tends to be higher thanthat of more complex, single die devices.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

To easily identify the discussion of any particular element or act, themost significant digit or digits in a reference number refer to thefigure number in which that element is first introduced.

FIG. 1A illustrates a first example of a chiplet system in accordancewith one embodiment.

FIG. 1B illustrates a second example of a chiplet system in accordancewith one embodiment.

FIG. 2 illustrates an example of a memory controller chiplet inaccordance with one embodiment.

FIG. 3 illustrates an SPI system in accordance with one embodiment.

FIG. 4 illustrates a first timing diagram in accordance with oneembodiment.

FIG. 5 illustrates a second timing diagram in accordance with oneembodiment.

FIG. 6 illustrates generally an example that includes identifying asecondary device status reply using information at an input port of aprimary device.

FIG. 7 illustrates generally an example that includes analyzingsecondary device status reply information.

FIG. 8 illustrates an example of a machine with which, in which, or bywhich embodiments of the present disclosure can operate.

DETAILED DESCRIPTION

A variety of communications protocols can be used to communicate betweena host and a memory device in a system, such as a chiplet system.Examples of such protocols can include the Open NAND Flash Interface(ONFi), eMMC, UFS, or Serial Peripheral Interface (SPI), among others.These protocols generally enable the host, primary device, orcontroller, to communicate commands—such as write (e.g., program), read,request the status of a command, request the status of the memorydevice, start or perform housekeeping operations such as intra-memorytransfers, garbage collection, etc.—with the memory device. Generally,these protocols restrict initiation of communication to the host. Thatis, the host makes a request and the memory device responds. In someexamples, the memory device can issue an exception (e.g., interrupt)that is designed to prompt the host to make a request for the status ofthe operation subject to the exception.

To facilitate communication among chiplets in a system, chiplets caninclude multiple input-output (I/O) channels (e.g., AIB channels), suchas can be arranged in columns of channels at a periphery of thechiplets. The I/O channels can be configured in a manner dependent onthe particular design or system objective. For example, the I/O channelsof chiplets can be configured as receive (RX) channels, transmit (TX)channels, or a mix of RX/TX channels.

In an example, one or more chiplets of a system can includeinitialization logic circuitry to advance a chiplet I/O interfacethrough various stages or phases of initialization. In some examples,chiplets can include a communication interface (e.g., a serialperipheral interface or SPI) and configuration data can be communicatedamong the chiplets using the communication interface. In some examples,the communication interface can be implemented using auxiliary channels(AUX) of the I/O channels and the configuration data can be communicatedusing out-of-band signaling.

In an example, initialization logic circuitry can be configured toadvance initialization of a chiplet interface sequentially through theinterface layers starting with a lowest interface layer (e.g., thephysical layer). The initialization can advance through multipleinitialization phases with one interface layer initialized during eachphase by writing initialization data to the chiplet I/O channels duringeach initialization phase.

Not all chiplet designs may have or use the same type of initialization.For example, individual chiplets may support only a hardware-basedinitialization option for the I/O channels or may support only asoftware-based initialization option for the I/O channels. Because achiplet-based system can include different chiplet designs mixed in thesame system, and without a standard method to support bothhardware-based and software-based approaches, each system would need anindividual, ad hoc hardware-based or software-based initializationmethod. This could result in some I/O channels of the chiplets not beinginteroperable with other I/O channels of the chiplets.

In an example, a standardized method of initialization of the I/Ochannels of the chiplets can allow all chiplets of a multi-chipletsystem and I/O interface to be interoperable within a single system. Thestandardized method can be used for both hardware-based andsoftware-based initialization mechanisms to guarantee interoperabilityof the interconnected chiplet I/O channels. In an example, theinitialization methods can include or use communication among chipletsusing an SPI bus. The SPI bus can be used to provide communicationbetween at least a primary device and a secondary device. As used inthis description, a primary device, such as a host, may manage orcontrol communications with or operations of one or more secondarydevices. The relationship between primary and secondary devices may bean asymmetric one that has sometimes been referred to in the art usingthe antiquated terms master and slave. Such relationships may also bereferred to as parent/child, supervisor/worker, controller/peripheral,or the like.

An issue with traditional SPI communications arises when a primarydevice, or primary chiplet, seeks to validate or confirm a presence orabsence of a secondary device, or selected chiplet, on the SPI bus. Inan example, without means to confirm a presence or absence of asecondary device during transactions between a primary device and thesecondary device, a primary device can be unaware of the success orfailure of various commands. For example, a primary device may be unableto determine whether a write command was delivered successfully, or thata read response or other response message is valid.

To address the issue, an SPI communication system (or other analogouscommunication system) can be configured to include or use amultiple-drop data communication channel between a primary device andmultiple secondary devices. The data communication channel can becoupled to a data input port of the primary device, and to respectivedata output ports of the multiple secondary devices. In an example, theprimary device can include pull-up circuitry such as a buffer, resistor,or other component, that is configured to maintain the data input portof the primary device in a defined state, such as unless or until thedata input port is driven by a signal from, for example, one of thesecondary devices. In an example, the defined state can include avoltage or current signal having a magnitude that is other than a logichigh or logic low signal magnitude in the system, sometimes referred toas a “weak” pull-up signal.

Systems and methods discussed herein can include or use the pull-upsignal at the data input port of the primary device to determine whethera primary device is present and active in the system. In an example, thesystems and methods can include or use a secondary device status fieldin messages communicated from respective secondary devices to theprimary device, and the primary device can use information in thesecondary device status field to determine whether a particulartransaction was successful. In an example, systems and methods discussedherein can be configured to probe and discover existing (i.e., present)and non-existing (i.e., absent) secondary devices or endpoints on thesystem.

In an example, another issue with traditional SPI communications arisesfrom a general dependence on timing or latency of peripherals orsecondary devices. Slower secondary devices, or secondary devicesconfigured to carry out computationally complicated or time-consumingactivities, can tie up or occupy an SPI bus and thus inhibitcommunication between other devices on the bus. For example, if an SPIsecondary device is unable to respond immediately to a read request froma primary device, then the SPI bus can be occupied unless or until thesecondary device completes its operations and prepares and sends itsresponse. This can be problematic and introduce significant delays, forexample, during initialization where multiple chiplets in a system needto be initialized to activate a system.

In an example, the system can be configured to use the secondary devicestatus field of response messages to help address the latency issue,such as by enabling deferred data transactions. In an example, a primarydevice, such as can include or comprise a first chiplet, can issue arequest for a specified payload. In response, a secondary device, or asecond chiplet, can be configured to generate a response message thatcan include a secondary device status field. The secondary device statusfield can include information about whether the secondary devicerequires or requests additional processing time to prepare a suitable orexpected reply, such as with the specified payload. The secondary devicecan send a message with the secondary device status field to the primarydevice and, in turn, the primary device can queue a later request, suchas for the same specified payload. In this manner, control overoccupation of the SPI bus can be provided to the primary device or firstchiplet. The primary device can then determine, for example, whether tocontinue issuing read requests for the specified payload to the samesecondary device, or whether other operations, such as using othersecondary devices coupled to the bus, can be performed.

FIG. 1A, described below, offers an example of a chiplet system and thecomponents operating therein. Within the context of such a chipletsystem, an issue can arise in communication among the chiplets or incommunication with other systems or devices coupled to the chipletsystem. In a chiplet system that includes a serial peripheral interface(SPI) bus or interface, a first peripheral device or a secondary devicegenerally can be configured to respond to a read request from acontroller or primary device within a specified number of clock cycles.The SPI interface, which is generally coupled to one or multiple othersecondary devices, can be inhibited from carrying out other datacommunication until the first secondary device, or first selectedchiplet, sends its complete response. The present inventors haverecognized, among other things, that a solution to this issue caninclude using a secondary device status field in an SPI message. Thesecondary device status field can indicate when a secondary device isready to send a response. If the secondary device status field indicatesthe secondary device is not ready to respond, then the primary devicecan be configured to issue a later request or deferred request. Thepresent inventors have further recognized, among other things, that theprimary device can be configured to use information from the secondarydevice status field to validate or confirm a transaction status. In anexample, the primary device can be configured to identify or confirm apresence or absence of one or more endpoints or secondary devices on theSPI bus, such as at initialization or at other times during operation,using information received at a data input port of the primary devicewhen a secondary device status field is expected. For example, theprimary device can be configured to sense its own pull-up behavior atits data input port when no secondary device endpoint is present on thedata channel that is coupled to the data input port of the primarydevice. Additional details and examples are provided below.

FIG. 1A and FIG. 1B illustrate an example of a first system 100 that caninclude one or more chiplets, according to an embodiment. FIG. 1A is arepresentation of the chiplet system 110 mounted on a peripheral board104, that can be connected to a broader computer system by a peripheralcomponent interconnect express (PCIe), for example. The chiplet system110 includes a package substrate 102, an interposer 120, and fourchiplets, an application chiplet 106, a host interface chiplet 112, amemory controller chiplet 114, and a memory device chiplet 118. Othersystems may include additional chiplets to provide additionalfunctionalities, as will be apparent from the following discussion. Thepackage of the chiplet system 110 is illustrated with a cover or lid126, though other packaging techniques and structures for the chipletsystem 110 can be used. FIG. 1B is a block diagram labeling thecomponents in the chiplet system for clarity.

The application chiplet 106 is illustrated as including anetwork-on-chip (NOC 108) to support an inter-chiplet communicationsnetwork, or chiplet network 122. In example embodiments, NOC 108 may beincluded on the application chiplet 106. In some examples, NOC 108 maybe defined in response to selected support chiplets (e.g., the hostinterface chiplet 112, memory controller chiplet 114, or memory devicechiplet 118) thus enabling a designer to select an appropriate number orchiplet network connections or switches for the NOC 108. In an example,the NOC 108 can be located on a separate chiplet, or even within theinterposer 120. In examples as discussed herein, the NOC 108 implementsan inter-chiplet communications network as a chiplet protocol interface(CPI) network.

The CPI is a packet-based network that supports virtual channels toenable a flexible and high-speed interaction between chiplets. CPIenables bridging from intra-chiplet networks to the chiplet network 122.For example, the Advanced eXtensible Interface (AXI) is a widely usedspecification to design intra-chip communications. AXI specifications,however, cover a great variety of physical design options, such as thenumber of physical channels, signal timing, power, etc. Within a singlechip, these options are generally selected to meet design goals, such aspower consumption, speed, etc. However, to achieve the flexibility ofthe chiplet system, an adapter, such as CPI, is used to interfacebetween the various AXI design options that can be implemented in thevarious chiplets. By enabling a physical channel to virtual channelmapping and encapsulating time-based signaling with a packetizedprotocol, CPI successfully bridges intra-chiplet networks across thechiplet network 122.

CPI can use a variety of different physical layers to transmit packets.The physical layer can include simple conductive connections, or caninclude drivers to increase the voltage, or otherwise facilitatetransmitting the signals over longer distances. An example of one suchphysical layer can include the Advanced Interface Bus (AIB), which invarious examples, can be implemented in the interposer 120. AIBtransmits and receives data using source synchronous data transfers witha forwarded clock. Packets are transferred across the AIB at single datarate (SDR) or dual data rate (DDR) with respect to the transmittedclock. Various channel widths are supported by AIB. AIB channel widthsare in multiples of 20 bits when operated in SDR mode (20, 40, 60, . . .), and multiples of 40 bits for DDR mode: (40, 80, 120, . . . ). The AIBchannel width includes both transmit and receive signals. The channelcan be configured to have a symmetrical number of transmit (TX) andreceive (RX) input/outputs (I/Os), or have a non-symmetrical number oftransmitters and receivers (e.g., either all transmitters or allreceivers). The channel can act as an AIB controller or peripheraldepending on which chiplet provides the controller clock. AIB I/O cellssupport three clocking modes: asynchronous (i.e. non-clocked), SDR, andDDR. In various examples, the non-clocked mode is used for clocks andsome control signals. The SDR mode can use dedicated SDR only I/O cells,or dual use SDR/DDR I/O cells.

In an example, CPI packet protocols (e.g., point-to-point or routable)can use symmetrical receive and transmit I/O cells within an AIBchannel. The CPI streaming protocol allows more flexible use of the AIBI/O cells. In an example, an AIB channel for streaming mode canconfigure the I/O cells as all TX, all RX, or half RX and half RX. CPIpacket protocols can use an AIB channel in either SDR or DDR operationmodes. In an example, the AIB channel is configured in increments of 80I/O cells (i.e. 40 TX and 40 RX) for SDR mode and 40 I/O cells for DDRmode. The CPI streaming protocol can use an AIB channel in either SDR orDDR operation modes. Here, in an example, the AIB channel is inincrements of 40 I/O cells for both SDR and DDR modes. In an example,each AIB channel is assigned a unique interface identifier. Theidentifier is used during CPI reset and initialization to determinepaired AIB channels across adjacent chiplets. In an example, theinterface identifier is a 20-bit value comprising a seven-bit chipletidentifier, a seven-bit column identifier, and a six-bit linkidentifier. The AIB physical layer transmits the interface identifierusing an AIB out-of-band shift register. The 20-bit interface identifieris transferred in both directions across an AIB interface using bits32-51 of the shift registers.

AIB defines a stacked set of AIB channels as an AIB channel column. AnAIB channel column has some number of AIB channels, plus an auxiliarychannel. The auxiliary channel contains signals used for AIBinitialization. All AIB channels (other than the auxiliary channel)within a column are of the same configuration (e.g., all TX, all RX, orhalf TX and half RX, as well as having the same number of data I/Osignals). In an example, AIB channels are numbered in continuousincreasing order starting with the AIB channel adjacent to the AUXchannel. The AIB channel adjacent to the AUX is defined to be AIBchannel zero.

Generally, CPI interfaces on individual chiplets can includeserialization-deserialization (SERDES) hardware. SERDES interconnectswork well for scenarios in which high-speed signaling with low signalcount are desirable. SERDES, however, can result in additional powerconsumption and longer latencies for multiplexing and demultiplexing,error detection or correction (e.g., using block level cyclic redundancychecking (CRC)), link-level retry, or forward error correction. However,when low latency or energy consumption is a primary concern forultra-short reach, chiplet-to-chiplet interconnects, a parallelinterface with clock rates that allow data transfer with minimal latencymay be utilized. CPI includes elements to minimize both latency andenergy consumption in these ultra-short reach chiplet interconnects.

For flow control, CPI employs a credit-based technique. A recipient,such as the application chiplet 106, provides a sender, such as thememory controller chiplet 114, with credits that represent availablebuffers. In an example, a CPI recipient includes a buffer for eachvirtual channel for a given time-unit of transmission. Thus, if the CPIrecipient supports five messages in time and a single virtual channel,the recipient has five buffers arranged in five rows (e.g., one row foreach unit time). If four virtual channels are supported, then therecipient has twenty buffers arranged in five rows. Each buffer holdsthe payload of one CPI packet.

When the sender transmits to the recipient, the sender decrements theavailable credits based on the transmission. Once all credits for therecipient are consumed, the sender stops sending packets to therecipient. This ensures that the recipient always has an availablebuffer to store the transmission.

As the recipient processes received packets and frees buffers, therecipient communicates the available buffer space back to the sender.This credit return can then be used by the sender to allow transmittingof additional information.

Also illustrated is a chiplet mesh network 124 that uses a direct,chiplet-to-chiplet technique without the need for the NOC 108. Thechiplet mesh network 124 can be implemented in CPI, or anotherchiplet-to-chiplet protocol. The chiplet mesh network 124 generallyenables a pipeline of chiplets where one chiplet serves as the interfaceto the pipeline while other chiplets in the pipeline interface only withthemselves.

Additionally, dedicated device interfaces, such as an SPI interface orone or more standard memory interfaces, such as the memory interface 116(such as, for example, synchronous memory interfaces, such as DDR5,DDR6), can also be used to interconnect chiplets. Connection of achiplet system or individual chiplets to external devices such as alarger system can be through a desired interface, for example, a PCIeinterface. Such an external interface may be implemented, in someexamples, through a host interface chiplet 112, which in the depictedexamples, provides a PCIe interface external to the chiplet system 110.Such dedicated interfaces are generally employed when a convention orstandard in the industry has converged on such an interface. Theillustrated example of a Double Data Rate (DDR) interface 116 connectingthe memory controller chiplet 114 to a dynamic random access memory(DRAM) memory device is an example of such an industry convention.

Of the variety of possible support chiplets, the memory controllerchiplet 114 is likely present in the chiplet system 110 due to the nearomnipresent use of storage for computer processing as well assophisticated state-of-the-art memory devices. Thus, using a memorydevice chiplet 118 and memory controller chiplet 114 produced by othersgives chiplet system designers access to robust products bysophisticated producers. Generally, the memory controller chiplet 114provides a memory device specific interface to read, write, or erasedata. Often, the memory controller chiplet 114 can provide additionalfeatures, such as error detection, error correction, maintenanceoperations, or atomic operation execution. For some types of memory,maintenance operations tend to be specific to the memory device chiplet118, such as garbage collection in NAND flash or storage class memories,temperature adjustments (e.g., cross temperature management) in NANDflash memories. In an example, the maintenance operations can includelogical-to-physical (L2P) mapping or management to provide a level ofindirection between the physical and logical representation of data. Insome flash memory configurations, for example, “managed NAND” devices,some or all of such management operations can be under control of adedicated NAND memory controller coupled to multiple NAND memory die. Inother types of memory, for example DRAM, some memory operations, such asrefresh, may be controlled by a host processor or by a memory controllerat some times, and at other times controlled by the DRAM memory deviceitself, or by logic associated with one or more DRAM devices, such as aninterface chip (in some examples, a buffer). Such an interface/buffermay be utilized in some examples to redistribute and change the clockrate of signals between an interface and individual memory devices. Insome examples, such an interface/buffer may incorporate additionalcontrol functionality.

Atomic operations are a data manipulation that, for example, may beperformed by the memory controller chiplet 114. In other chipletsystems, the atomic operations may be performed by other chiplets. Forexample, an atomic operation of “increment” can be specified in acommand by the application chiplet 106, the command including a memoryaddress and possibly an increment value. Upon receiving the command, thememory controller chiplet 114 retrieves a number from the specifiedmemory address, increments the number by the amount specified in thecommand, and stores the result. Upon a successful completion, the memorycontroller chiplet 114 provides an indication of a command success tothe application chiplet 106. Atomic operations avoid transmitting thedata across the chiplet network 122, resulting in lower latencyexecution of such commands.

Atomic operations can be classified as built-in atomics or programmable(e.g., custom) atomics. Built-in atomics are a finite set of operationsthat are immutably implemented in hardware. Programmable atomics aresmall programs that can run on a programmable atomic unit (PAU) (e.g., acustom atomic unit (CAU)) of the memory controller chiplet 114. FIG. 1Aillustrates an example of a memory controller chiplet that discusses aPAU.

The memory device chiplet 118 can be, or include any combination of,volatile memory devices or non-volatile memories. Examples of volatilememory devices include, but are not limited to, random access memory(RAM)—such as DRAM, synchronous DRAM (SDRAM), graphics double data ratetype 6 SDRAM (GDDR6 SDRAM), among others. Examples of non-volatilememory devices include, but are not limited to, negative-and-(NAND)-typeflash memory, storage class memory (e.g., phase-change memory ormemristor based technologies), ferroelectric RAM (FeRAM), among others.The illustrated example includes the memory device chiplet 118 as achiplet, however, the memory device chiplet 118 can reside elsewhere,such as in a different package on the peripheral board 104. For manyapplications, multiple memory device chiplets may be provided. In someexamples, these memory device chiplets may each implement one ormultiple storage technologies. In some examples, a memory chiplet mayinclude, multiple stacked memory die of different technologies, forexample one or more SRAM devices stacked or otherwise in communicationwith one or more DRAM devices. Memory controller chiplet 114 may alsoserve to coordinate operations between multiple memory chiplets in thechiplet system 110; for example, to utilize one or more memory chipletsin one or more levels of cache storage, and to use one or moreadditional memory chiplets as main memory. Chiplet system 110 may alsoinclude multiple memory controllers, as may be used to provide memorycontrol functionality for separate processors, sensors, networks, etc. Achiplet architecture, such as in the chiplet system 110 offersparticular advantages in allowing adaptation to different memory storagetechnologies, and different memory interfaces, through updated chipletconfigurations, without requiring redesign of the remainder of thesystem structure.

FIG. 2 illustrates components of an example of a memory controllerchiplet 218, such as the memory controller chiplet 114 of FIG. 1A,according to an embodiment. The memory controller chiplet 218 includes acache 202, a cache controller 204, an off-die memory controller 206(e.g., to communicate with an off-die memory 230), a networkcommunication interface 208 (e.g., to interface with the chiplet network122) and communicate with other chiplets), an SPI controller 232, and aset of atomic and merge operations 220. Members of this set can include,for example, a write merge unit 222, a hazard unit (memory hazard clearunit 224), built-in atomic unit 226, or a PAU 228. The variouscomponents are illustrated logically, and not as they necessarily wouldbe implemented. For example, the built-in atomic unit 226 likelycomprises different devices along a path to the off-die memory. Forexample, the built-in atomic unit 226 could be located in an interfacedevice/buffer on a memory chiplet, as discussed above. In contrast, theprogrammable atomic operations are likely implemented in a separateprocessor on the memory controller chiplet 218 (but in various examplesmay be implemented in other locations, for example on a memory chiplet).

The off-die memory controller 206 is directly coupled to the off-diememory 230 (e.g., via a bus or other communication connection) toprovide write operations and read operations to and from the off-diememory 230. In the depicted example, the off-die memory controller 206is also coupled for output to the atomic and merge operations 220, andfor input to the cache controller 204 (e.g., a memory side cachecontroller). In an example, the off-die memory controller 206 can becoupled to the off-die memory 230 using an SPI bus.

In an example, the off-die memory controller 206 (e.g., a memorycontroller for off-die memory) can include or comprise a portion of theSPI controller 232. The SPI controller 232 can be coupled to an SPI busand configured to manage communication between the memory controllerchiplet 114 and one or more other chiplets, such as other chiplets inthe chiplet network 122 or the off-die memory 230. In an example, thememory controller chiplet 114 can use the SPI controller 232 to carryout initialization routines with various chiplets coupled to the memorycontroller chiplet 114. Once initialized, the memory controller chiplet114 can continue to use SPI-based communications with the chiplets orcan change to using other protocols or busses.

In an example, the SPI controller 232 or bus controller can be providedon the host interface chiplet 112, and the host interface chiplet 112can use a PCIe interface to communicate outside of the chiplet system110. A memory controller, such as the memory controller chiplet 114, canbe an SPI memory device or SPI secondary device. The memory controllercan, in turn, be configured to use another memory interface such as thememory interface 116.

In the example configuration, the cache controller 204 is directlycoupled to the cache 202, and may be coupled to the networkcommunication interface 208 for input (such as incoming read or writerequests), and coupled for output to the off-die memory controller 206.

The network communication interface 208 includes a packet decoder 210,network input queues 212, a packet encoder 214, and network outputqueues 216 to support a packet-based chiplet network 122, such as CPI.The chiplet network 122 can provide packet routing between and amongprocessors, memory controllers, hybrid threading processors,configurable processing circuits, or communication interfaces. In such apacket-based communication system, each packet typically includesdestination and source addressing, along with any data payload orinstruction. In an example, the chiplet network 122 can be implementedas a collection of crossbar switches having a folded clos configuration,or a mesh network providing for additional connections, depending uponthe configuration.

In various examples, the chiplet network 122 can be part of anasynchronous switching fabric. Here, a data packet can be routed alongany of various paths, such that the arrival of any selected data packetat an addressed destination can occur at any of multiple differenttimes, depending upon the routing. Additionally, the chiplet network 122can be implemented at least in part as a synchronous communicationnetwork, such as a synchronous mesh communication network. Bothconfigurations of communication networks are contemplated for use inaccordance with the present disclosure.

The memory controller chiplet 218 can receive a packet having, forexample, a source address, a read request, and a physical address. Inresponse, the off-die memory controller 206 or the cache controller 204will read the data from the specified physical address (which can be inthe off-die memory 230 or in the cache 202), and assemble a responsepacket to the source address containing the requested data. Similarly,the memory controller chiplet 218 can receive a packet having a sourceaddress, a write request, and a physical address. In response, thememory controller chiplet 218 will write the data to the specifiedphysical address (which can be in the off-die memory 230 or in the cache202), and assemble a response packet to the source address containing anacknowledgement that the data was stored to a memory.

Thus, the memory controller chiplet 218 can receive read and writerequests via the chiplet network 122 and process the requests using thecache controller 204 interfacing with the cache 202, if possible. If therequest cannot be handled by the cache controller 204, then the off-diememory controller 206 handles the request by communication with theoff-die memory 230, the atomic and merge operations 220, or both. Asnoted above, one or more levels of cache may also be implemented inoff-die memory 230 and in some such examples may be accessed directly bythe cache controller 204. Data read by the off-die memory controller 206can be cached in the cache 202 by the cache controller 204 for lateruse.

The atomic and merge operations 250 are coupled to receive (as input)the output of the off-die memory controller 220, and to provide outputto the cache 210, the network communication interface 225, or directlyto the chiplet network 280. The reset or memory hazard clear unit 224,write merge unit 222, and the built-in (e.g., predetermined) built-inatomic unit 226 can each be implemented as state machines with othercombinational logic circuitry (such as adders, shifters, comparators,AND gates, OR gates, XOR gates, or any suitable combination thereof) orother logic circuitry. These components can also include one or moreregisters or buffers to store operand or other data. The PAU 228 can beimplemented as one or more processor cores or control circuitry, andvarious state machines with other combinational logic circuitry or otherlogic circuitry, and can also include one or more registers, buffers, ormemories to store addresses, executable instructions, operand and otherdata, or can be implemented as a processor.

The write merge unit 222 receives read data and request data, and mergesthe request data and read data to create a single unit having the readdata and the source address to be used in the response or return datapacket). The write merge unit 222 provides the merged data to the writeport of the cache 202 (or, equivalently, to the cache controller 204 towrite to the cache 202). Optionally, the write merge unit 222 providesthe merged data to the network communication interface 208 to encode andprepare a response or return data packet for transmission on the chipletnetwork 122.

When the request data is for a built-in atomic operation, the built-inatomic unit 226 receives the request and reads data, either from thewrite merge unit 222 or directly from the off-die memory controller 206.The atomic operation is performed, and using the write merge unit 222,the resulting data is written to the cache 202, or provided to thenetwork communication interface 208 to encode and prepare a response orreturn data packet for transmission on the chiplet network 122.

The built-in atomic unit 226 handles predefined atomic operations suchas fetch-and-increment or compare-and-swap. In an example, theseoperations perform a simple read-modify-write operation to a singlememory location of 32-bytes or less in size. Atomic memory operationsare initiated from a request packet transmitted over the chiplet network122. The request packet has a physical address, atomic operator type,operand size, and optionally up to 32-bytes of data. The atomicoperation performs the read-modify-write to a cache memory line of thecache 202, filling the cache memory if necessary. The atomic operatorresponse can be a simple completion response, or a response with up to32-bytes of data. Example atomic memory operators include fetch-and-AND,fetch-and-OR, fetch-and-XOR, fetch-and-add, fetch-and-subtract,fetch-and-increment, fetch-and-decrement, fetch-and-minimum,fetch-and-maximum, fetch-and-swap, and compare-and-swap. In variousexample embodiments, 32-bit and 64-bit operations are supported, alongwith operations on 16 or 32 bytes of data. Methods disclosed herein arealso compatible with hardware supporting larger or smaller operationsand more or less data.

Built-in atomic operations can also involve requests for a “standard”atomic operation on the requested data, such as a comparatively simple,single cycle, integer atomics-such as fetch-and-increment orcompare-and-swap-which will occur with the same throughput as a regularmemory read or write operation not involving an atomic operation. Forthese operations, the cache controller 204 may generally reserve a cacheline in the cache 202 by setting a hazard bit (in hardware), so that thecache line cannot be read by another process while it is in transition.The data is obtained from either the off-die memory 230 or the cache202, and is provided to the built-in atomic unit 226 to perform therequested atomic operation. Following the atomic operation, in additionto providing the resulting data to the packet encoder 214 to encodeoutgoing data packets for transmission on the chiplet network 122, thebuilt-in atomic unit 226 provides the resulting data to the write mergeunit 222, which will also write the resulting data to the cache 202.Following the writing of the resulting data to the cache 202, anycorresponding hazard bit which was set will be cleared by the memoryhazard clear unit 224.

The PAU 228 enables high performance (high throughput and low latency)for programmable atomic operations (also referred to as “custom atomicoperations”), comparable to the performance of built-in atomicoperations. Rather than executing multiple memory accesses, in responseto an atomic operation request designating a programmable atomicoperation and a memory address, circuitry in the memory controllerchiplet 218 transfers the atomic operation request to PAU 228 and sets ahazard bit stored in a memory hazard register corresponding to thememory address of the memory line used in the atomic operation, toensure that no other operation (read, write, or atomic) is performed onthat memory line, which hazard bit is then cleared upon completion ofthe atomic operation. Additional, direct data paths provided for the PAU228 executing the programmable atomic operations allow for additionalwrite operations without any limitations imposed by the bandwidth of thecommunication networks and without increasing any congestion of thecommunication networks.

The PAU 228 includes a multi-threaded processor, for example, such as aRISC-V ISA based multi-threaded processor having one or more processorcores, and further having an extended instruction set for executingprogrammable atomic operations. When provided with the extendedinstruction set for executing programmable atomic operations, the PAU228 can be embodied as one or more hybrid threading processors. In someexample embodiments, the PAU 228 provides barrel-style, round-robininstantaneous thread switching to maintain a high instruction-per-clockrate.

Programmable atomic operations can be performed by the PAU 228 involvingrequests for a programmable atomic operation on the requested data. Auser can prepare programming code to provide such programmable atomicoperations. For example, the programmable atomic operations can becomparatively simple, multi-cycle operations such as floating-pointaddition, or comparatively complex, multi-instruction operations such asa Bloom filter insert. The programmable atomic operations can be thesame as or different than the predetermined atomic operations, insofaras they are defined by the user rather than a system vendor. For theseoperations, the cache controller 204 can reserve a cache line in thecache 202, by setting a hazard bit (in hardware), so that cache linecannot be read by another process while it is in transition. The data isobtained from either the off-die memory 230 or the cache 202, and isprovided to the PAU 228 to perform the requested programmable atomicoperation. Following the atomic operation, the PAU 228 will provide theresulting data to the network communication interface 208 to directlyencode outgoing data packets having the resulting data for transmissionon the chiplet network 122. In addition, the PAU 228 will provide theresulting data to the cache controller 204, which will also write theresulting data to the cache 202. Following the writing of the resultingdata to the cache 202, any corresponding hazard bit which was set willbe cleared by the cache controller 204.

In selected examples, the approach taken for programmable atomicoperations is to provide multiple, generic, custom atomic request typesthat can be sent through the chiplet network 122 to the memorycontroller chiplet 218 from an originating source such as a processor orother system component. The cache controller 204 and/or off-die memorycontroller 206 identify the request as a custom atomic and forward therequest to the PAU 228. In a representative embodiment, the PAU 228: (1)is a programmable processing element capable of efficiently performing auser defined atomic operation; (2) can perform load and stores tomemory, arithmetic and logical operations and control flow decisions;and (3) leverages the RISC-V ISA with a set of new, specializedinstructions to facilitate interacting with the controllers toatomically perform the user-defined operation. In desirable examples,the RISC-V ISA contains a full set of instructions that support highlevel language operators and data types. The PAU 228 can leverage theRISC-V ISA, but will commonly support a more limited set of instructionsand limited register file size to reduce the die size of the unit whenincluded within the memory controller chiplet 218.

As mentioned above, prior to the writing of the read data to the cache202, the set hazard bit for the reserved cache line is to be cleared, bythe memory hazard clear unit 224. Accordingly, when the request and readdata is received by the write merge unit 222, a reset or clear signalcan be transmitted by the memory hazard clear unit 224 to the cache 202to reset the set memory hazard bit for the reserved cache line. Also,resetting this hazard bit will also release a pending read or writerequest involving the designated (or reserved) cache line, providing thepending read or write request to an inbound request multiplexer forselection and processing.

In an example, a chiplet system can be configured to include or use aserial peripheral interface (SPI) bus or interface. An SPI bus generallyincludes a multiple-wire serial communication interface that enables asynchronous data link between two devices, a primary device and asecondary device. It provides support for a low to medium bandwidthnetwork connection between the devices supporting the SPI. In a chipletsystem with an SPI bus, the primary device can be a first chiplet, andthe secondary device, or secondary devices, can be one or more otherchiplets coupled to the first chiplet using the SPI bus.

The wires of the SPI bus typically include two control channels and twodata channels. The control channels of the SPI bus include a chip select(CS) channel, and a serial clock channel (SCLK). In some examples, morethan one secondary device can be connected to the primary device,however, only one secondary device can generally be accessed at anygiven time. Selection of a particular secondary device from amongmultiple secondary devices can be accomplished using the chip select orCS channel. A CS signal is outputted from the primary device and allowsfor activation and deactivation of a secondary device by the primarydevice. In an example, every secondary device can use its own separateCS channel for activation. In other examples discussed herein, a singlemultiple-drop CS channel can be used for global activation ordeactivation of multiple secondary devices, and unique secondary deviceidentifiers can be used to select a particular secondary device toreceive a command from the primary device.

The primary device can also provide a serial clock signal. The clocksignal can be used to clock the shifting of serial data into and out ofthe primary and secondary devices. Use of this clock allows SPIarchitecture to operate in a primary/secondary full duplex mode, whereindata can be transferred simultaneously from one device to another.

The data channels of the SPI bus can include a Serial Data In (SDI) lineand a Serial Data Out (SDO) line. The SDI line is a data communicationline that outputs data from the primary device to the secondary device.The SDO line is a second data communication line used to output datafrom the secondary device to the primary device. Both data channels areactive when the chip select channel is activated for the specifiedsecondary device, unless the specified secondary device is configured tooperate in an always-on manner.

In an example, initialization of I/O channels of multiple chiplets inthe chiplet system 110 can include or use an SPI bus. Using the SPI bus,I/O channels for all chiplets of a multi-chiplet I/O interface can beinteroperable within a single system. The SPI-based techniques discussedherein can be used for both hardware-based and software-basedinitialization mechanisms to guarantee interoperability of theinterconnected chiplet I/O channels.

FIG. 3 illustrates generally an example of an SPI system 300. The SPIsystem 300 can include a primary device 312 or first chiplet interfacedwith a secondary device or second chiplet, such as an SPI memory device302. In an example, the SPI system 300 can comprise a portion of thefirst system 100 from the example of FIG. 1A, or the first example 200from the example of FIG. 2. For example, one or more of the primarydevice 312 and the SPI memory device 302 can be a respective chiplet inthe first system 100. In an example, the first system 100 includes theprimary device 312, the SPI memory device 302 resides outside of thefirst system 100, and the primary device 312 communicates with the SPImemory device 302 using an SPI interface.

In an example, the primary device 312 comprises the memory controllerchiplet 114 and the SPI controller 232, and the off-die memory 230comprises the SPI memory device 302. The primary device 312 can includeports for coupling to respective channels or busses in an SPI interface.For example, the primary device 312 can include a data input port 338configured to receive data signals from a data communication channel, adata output port 340 configured to provide data signals to a differentdata communication channel, a chip select signal port 342 configured toprovide a chip select signal to one or multiple secondary devices usinga chip select signal bus, and a clock signal port 344 configured toprovide a system clock signal to a clock signal channel.

In an example, one or more of the ports of the primary device 312 caninclude or use buffer circuitry, such as tri-state buffer circuitry. Forexample, the data input port 338 or the data output port 340 can includerespective tri-state buffer circuitry. The buffer circuitry can beprovided internally to the primary device 312 or can be external. In anexample, the data input port 338 is coupled to pull-up circuitry that isconfigured to maintain a fixed voltage or current at the data input port338, such as when the bus or channel coupled to the data input port 338is unused or unoccupied, such as between data transactions or when nosecondary device on the bus is selected by the primary device 312.

A peripheral device or secondary device, such as the SPI memory device302, can include a secondary device controller 314 with multipleinterface ports or pins, including a chip select (CS) port to receive aselect signal 318, a clock (SCLK) port to receive a clock signal 320, acontroller output peripheral input port (COPI port or MOSI port) port toreceive a secondary device input signal 322 from the primary device 312,and a controller input peripheral output port (CIPO port or MISO port)to provide a secondary device output signal 316 to the primary device312. Data transfer between the primary device 312 and the SPI memorydevice 302 or secondary device can take place serially and synchronouslyusing the secondary device output signal 316 and the secondary deviceinput signal 322. In an example, the CIPO port of the SPI memory device302 can be configured to include or use tri-state buffer circuitry. Thetri-state buffer circuitry can be configured to maintain the CIPO portin a high impedance state, or an effectively electrically disconnectedstate, when the SPI memory device 302 is not selected for communicationwith the primary device 312 (e.g., selected according to a signal on theCS port). When the SPI memory device 302 is selected for communication,then the SPI memory device 302 and buffer circuitry can be configured touse logic-level communication signals (e.g., high/low signals) suitablefor the given system.

In an example, the SPI memory device 302 can include a device identifier310. The device identifier 310 can include a memory address or registerthat can be statically or dynamically assigned to a particular value,such as in response to a Static ID input. That is, the device identifier310 can be statically or dynamically strapped to a value that uniquelyidentifies the SPI memory device 302, such as when the SPI memory device302 is used together with other secondary devices in the SPI system 300.

In an example, the primary device 312 can enable the secondary devicecontroller 314 by setting the chip select signal 318 low. In an example,the secondary device controller 314 can be enabled using astatically-tied chip select port at the SPI memory device 302, such aswhen a CS signal is unused or unavailable. After the secondary devicecontroller 314 is enabled, the primary device 312 can send the clocksignal 320 and a corresponding data signal, or secondary device inputsignal 322. Each bit transmitted in the secondary device input signal322 or secondary device output signal 316 can be synchronous to either arising edge or falling edge of the clock signal 320. In an example, theprimary device 312 can provide data on the secondary device input signal322 latched on a rising clock edge and the SPI memory device 302 canprovide data on the secondary device output signal 316 on a fallingedge. In this example, a first rising edge of the clock signal 320 cancorrespond to the first bit of the secondary device input signal 322,and subsequent rising clock edges of the clock signal 320 can correspondto subsequent bits of the secondary device input signal 322. Similarly,each bit output on the secondary device output signal 316 can transitionon a falling edge of the clock signal 320. Other signal timing schemescan similarly be used.

In an example, communication between the primary device 312 and thesecondary device controller 314 can begin when the primary device 312sets the select signal 318 low. The primary device 312 can subsequentlysend the clock signal 320 and begin transmitting a message using thesecondary device input signal 322. In an example, a message can includea one-byte command followed by a memory address of one or more wholebytes, and further followed by data of one or more whole bytes. Thesecondary device controller 314 can respond by sending a synchronousmessage using the secondary device output signal 316. Due to the natureof conventional SPI, the secondary device controller 314 can beconfigured to output information on the secondary device output signal316 until a specified or expected time at which the primary device 312expects or is configured to receive a response.

In an example, the primary device 312 can send a write register commandor a read register command in a message to the secondary devicecontroller 314. The write register command or read register command canconfigure the secondary device controller 314 to access memory registers324. Data transfer to and from the secondary device controller 314 andregisters 324 can use a register bus 334, such as can have a dedicatedcontrol wire. The registers 324 can include, for example, a statusregister to indicate device operation status and/or a special modeenable register such as a block writing lock register or BWL register326 such as to prevent certain portions of memory from being written. Inan example, the registers 324 can include a one time programmable (OTP)enable register, or OTP register 328, such as to enable reading from orwriting to an OTP portion of memory, and/or a parameter page (PP) enableregister, or PP register 330, such as to enable reading from or writingto a parameter page of memory. In an example, the secondary devicecontroller 314 can be configured to access the registers 324 when itperforms other internal operations.

In an example, access to the registers 324 can permit a user or clientto control functional aspects of the SPI memory device 302, such as anoutput buffer drive strength, a desired number of clock cycles oflatency for outputting data, an address cycle format to require wholebytes or to use a minimum number of addresses, and/or whether to enableor disable error correcting codes (ECC). Certain registers can hold, forexample, error status information, such as can be reset upon theissuance of a register write command, while other registers can enable auser to control timing based on varying clock signal 320 frequencies. Inan example, a register can be configured to enable the SPI memory device302 to switch between different modes and interfaces, such as betweenSPI NAND and NAND user modes and interfaces.

In an example, to perform an operation at a specified memory address,the secondary device controller 314 can send a memory address signalusing a bus to row and column decoders (not depicted). The secondarydevice controller 314 can control activation of the row and columndecoders using a control wire. Depending on the operation, the secondarydevice controller 314 can, for example, load data bytes into a cacheregister 304 using a cache bus 332. In an example, a NAND array 308 canreceive data, such as one page at a time, through a data register 306,such as in coordination with the cache register 304.

In an example, a first SPI peripheral or first SPI secondary device,such as the SPI memory device 302 or other secondary device or chiplet,can be configured to respond to a read request from a controller orother chiplet, such as the primary device 312, within a specified numberof clock cycles. The SPI bus, such as can be coupled to one or multiplesecondary devices or chiplets, can be impeded from carrying out otherdata communication until the first SPI peripheral or secondary devicesends a complete response to the primary device 312. The presentinventors have recognized, among other things, that a solution to thisbus occupation issue can include using a secondary device status fieldin an SPI message that is sent from the first SPI secondary device. Forexample, the secondary device status field can be included in thesecondary device output signal 316 that is communicated to the primarydevice 312. The secondary device status field can indicate that thesending secondary device is or is not ready to send a response. When thesecondary device status field indicates the secondary device is notready to send a response, then the primary device 312 can be configuredto issue a later or deferred request to the secondary device. Systemsand methods discussed herein can thus be used to enable long latency SPIoperations to proceed in the background while an SPI bus is used forother communications or transactions, such as between the primary device312 and one or more other secondary devices. In an example, systems andmethods discussed herein can be used during initialization routines forchiplet systems, for example, using SPI operations and hardware forinitial communications before transitioning to other protocols orbusses.

In an example, the primary device 312 can be configured to monitor aCIPO port, such as the data input port 338, for a message that includesa secondary device status field. In an example, a data communicationchannel coupled to the CIPO port can be unused or unoccupied during aclock period, or specified number of clock signal cycles, thatcorresponds to a response period or time when a response message isexpected by the primary device 312. In other words, the primary device312 can expect a response message from a secondary device, such as theSPI memory device 302, within a particular number of cycles of the clocksignal 320, and the response message can be configured to include asecondary device status field. In an example that does not include aresponse message, the primary device 312 can be configured to read orinterpret a pull-up signal at the data input port 338 as informationabout the secondary device status. If the pull-up signal is high for aspecified number of clock cycles (e.g., one, two, or more cyclesdepending on the configuration of the system) then the primary device312 can determine that a secondary device endpoint is absent from thechannel.

In an example, when a secondary device responds with a deferred readresponse status, the primary device 312 can queue a later read requestfor the deferred information. Between receiving the deferred readresponse status information from the secondary device and issuing thelater deferred read request, the primary device 312 can perform otheroperations using the same SPI bus. In an example, the primary device 312can eventually issue a deferred read request, such as using a specialrequest signal. In an example, the primary device 312 can immediatelyissue the deferred read request upon receipt of the deferred readresponse from a secondary device, or the primary device 312 can issuethe deferred read request after a specified or variable blanking period.

In an example, in response to the primary device 312 issuing a deferredread request using the special request signal, the secondary device canoptionally issue another deferral if the payload is not yet available.If the payload is available, then the secondary device can reply with amessage that includes a successful secondary device status field and thepayload.

The examples of FIG. 4 and FIG. 5 illustrate generally examplecommunication methods or timing diagrams that can include or usesecondary device identifiers and can be used for carrying outdeferred-response communications using an SPI bus. The timing diagramsdescribe communications using multiple different signal channels orbuses on the SPI bus. Table 1 describes the conventions used in thetiming diagrams for the illustrated message fields and associatedmessage contents or usage.

TABLE I Example Message Fields in SPI Deferred-Response CommunicationsMessage Field Usage C[1:0] 2-bit Command ID[6:0] 7-bit ChipletIdentification IDP Command and Chiplet Identification; e.g., Odd ParityA[31:0] 32-bit Address AP Address; e.g., Odd Parity D[63:0] 64-bit Reador Write Data DP Data; e.g., Odd Parity S[2:0] Secondary Device StatusSP Secondary Device Status; e.g., Odd Parity

In an example, a message sent from a primary device to a secondarydevice can include a 2-bit Command field (e.g., C[1:0] in Table 1). TheCommand field can comprise a portion of the secondary device inputsignal 322 and can indicate a command or instruction from the primarydevice 312. In an example, the Command field can include informationabout whether the receiving device or secondary device is directed toperform a read operation or a write operation. In an example, theCommand field can indicate whether a controller request (e.g., a readrequest) is an initial request or a deferred request.

In an example, a message can include a 7-bit Chiplet Identificationfield and parity bit (e.g., ID[6:0] and IDP in Table 1). The ChipletIdentification field can include an identifier that can be used, forexample, to address a particular chiplet or particular secondary devicein a system, such as in the chiplet system 110. In an example, a ChipletIdentification field can be used when peripheral or secondary device SPIchip select channels are unavailable or are unused in an SPI bus. In anexample, a Chiplet Identification field can be used to reduce oreliminate a number of chip select channels or lines that are required orused in an SPI bus.

In an example, a 32-bit Address and parity bit (e.g., A[31:0] and AP inTable 1) can follow the Chiplet Identification message. The Addressmessage can be used, for example, to locate a particular register, suchas in the data register 306 or elsewhere in the chiplet system 110. Inan example, a 64-bit Data message and parity bit (e.g., D[63:0] and DPin Table 1) can follow the Address message. The Data message cancomprise a data payload such as for storage in, or retrieval from, thesecondary device.

In an example, a message that includes a secondary device status fieldand parity bit (e.g., S[2:0] and SP in Table 1) can be provided from thesecondary device to the primary device. The secondary device statusfield can, in an example, be a one, two, three, or more bit field ormessage. In the examples illustrated herein, the secondary device statusfield comprises a 3-bit message and a parity bit, however secondarydevice status fields or messages can be configured to have more or fewerbits depending on an amount of information to be exchanged. Variouscommands or information can be encoded in a secondary device statusfield, such as to indicate a successful operation, an unsuccessful orincomplete operation, or a fault. Table 2 describes generally variouscommands or information that can be encoded in a Secondary device statusmessage.

TABLE 2 Secondary device status Message Commands S[2:0] Command 0Success; e.g., a CS R operation completed successfully 1 Deferred ReadResponse; e.g., a CSR Read operation was not completed 2 Odd ParityError; e.g., a CSR operation was aborted 3 Protocol Error; e.g., a CSRoperation was aborted 4-6 Reserved 7 No Secondary Device EndpointPresent

The various message fields, usages, and message components discussedherein are examples only and should not be considered limiting. Forexample, other additional message fields can be used, or fewer messagefields can be used in SPI deferred-response communications. In anexample, the various fields can be arranged in different orders orsequences to similarly provide deferred-response communications. Thevarious sizes or bit lengths of the fields or message components areprovided as examples only.

Using the conventions provided in Table 1, FIG. 4 illustrates generallya first timing diagram 400 that includes using a secondary deviceidentification field 408 for a read operation, and FIG. 5 illustratesgenerally a second timing diagram 500 that includes using a secondarydevice status field 508 for a write operation.

The example of FIG. 4 shows a general timing diagram for various signalscommunicated using an SPI bus, such as between the primary device 312and the SPI memory device 302. FIG. 4 includes examples of the selectsignal 318, the clock signal 320, the secondary device input signal 322,and the secondary device output signal 316, such as on separate channelsin an SPI bus. As mentioned above, the primary device 312 can initiatecommunication by setting the select signal 318 low, as generallyindicated by reference numeral 402.

In the example of FIG. 4, a first rising edge 404 of the clock signal320 corresponds to a first bit of the 2-bit Command field. In FIG. 4,the 2-bit Command field is 0-1 and indicates to the secondary devicethat the message includes a first or initial read instruction. Followingthe Command field, the primary device 312 can send the ChipletIdentification field portion of the message. The Chiplet Identificationfield can include an n-bit identifier of a specific one of multiplesecondary devices present on the SPI bus or in the system. The ChipletIdentification field can be followed by the Address field, such as toindicate a register location. The bits of the various message bitcomponents can correspond to respective pulses in the clock signal 320.In the example of FIG. 4, a blanking period can follow the Addressfield.

In response to the Command, Chiplet Identification, and Address fields,the secondary device can prepare and communicate a response to theprimary device 312, such as using the secondary device output signal316. In the example of FIG. 4, the secondary device output signal 316comprises a signal that leads with a secondary device status field 410.Depending on the information in the secondary device status field 410,the secondary device output signal 316 can include or comprise apayload, such as comprising an n-bit Data field. The communication canterminate when the primary device 312 sets the select signal 318 high,such as indicated in FIG. 4 by reference numeral 412.

In an example, the secondary device output signal 316 can be unavailableor absent, such as when the particular secondary device that isaddressed or selected by the primary device 312 is inoperative orunused. If the particular secondary device is absent, then it isunavailable to drive the CIPO port or data input port 338 of the primarydevice 312 in response to the read message in the secondary device inputsignal 322. In an example, the primary device 312 can include or usepull-up circuitry at the data input port 338 to hold the CIPO port in anon-zero state, including when the secondary device is absent from thedata bus. Accordingly, the primary device 312 can be configured to readthe non-zero state information from the data input port 338 due to thepull-up circuitry, such as corresponding to the secondary deviceidentification field 408, such as during clock cycles that followpresentation of the read message. The primary device 312 can interpretthe non-zero state information as a portion of the secondary devicestatus field, such as can indicate that no secondary device endpoint ispresent when the non-zero state information represents a high signalover a specified number of clock cycles (e.g., when the secondary deviceidentification field 408 is 1-1-1, such as when secondary device statusfield is configured to include three bits).

The example of FIG. 5 shows a second timing diagram 500 for varioussignals communicated using an SPI bus, such as between the primarydevice 312 and the SPI memory device 302. FIG. 5 includes examples ofthe select signal 318, the clock signal 320, the secondary device inputsignal 322, and the secondary device output signal 316. As mentionedabove, the primary device 312 can initiate communication by setting theselect signal 318 signal low, as generally indicated by referencenumeral 502.

In the example of FIG. 5, a first rising edge 504 of the clock signal320 corresponds to a first bit of the 2-bit Command field. In FIG. 5,the 2-bit Command field is 1-0 and indicates to the secondary devicethat the message includes a write instruction. Following the Commandfield, the primary device 312 can send the Chiplet Identification field,such as followed by the Address field, such as to indicate a particularsecondary device and a register location on the particular secondarydevice. Following the Address field, the primary device 312 can send aData field or payload, such as for storage in a register indicated bythe information in the Address field portion of the communication. Inthe example of FIG. 5, a blanking period can follow the Data field.

Following the blanking period and in response to the write instruction,the secondary device can return a message to the primary device 312using the secondary device output signal 316. In the example of FIG. 5,the returned message can include a secondary device status field 508.Using the information in the secondary device status field 508, theprimary device 312 can be configured to take a particular responsiveaction. For example, the secondary device status field 508 can indicateto the primary device 312 that it should queue a deferred read request,such as to the same or different secondary device. Various other actionscan be instructed using the secondary device status field 508, such asdescribed above in Table 2, among others.

In an example, the secondary device output signal 316 can be unavailableor absent, such as when the particular secondary device that isaddressed or selected by the primary device 312 is unavailable,inoperative or unused. If the particular secondary device is absent,then it is unavailable to drive the CIPO port or data input port 338 ofthe primary device 312 in response to the read message in the secondarydevice input signal 322. Owing to pull-up circuitry at the data inputport 338 of the primary device 312, the data input port 338 can bemaintained in a non-zero state, including when a secondary device isabsent from the data bus. Accordingly, the primary device 312 can beconfigured to read the non-zero state information from the data inputport 338, such as corresponding to the secondary device status field508, following the write message from the primary device 312. Theprimary device 312 can interpret the non-zero state information as anindication that no secondary device endpoint is present when thenon-zero state information represents a high signal over a specifiednumber of clock cycles (e.g., when the secondary device status field 508is 1-1-1, such as when the secondary device status field is configuredto include three bits).

FIG. 6 illustrates generally a flow diagram of an example of a firstmethod 600 for communication using an SPI interface and a data messagethat includes secondary device status information. In the example ofFIG. 6, the first method 600 begins at block 602 with providing apull-up signal at a data input port of a first chiplet. In an example,block 602 can include providing the pull-up signal using buffercircuitry at the data input port 338 of the primary device 312.

At block 604, the first method 600 can include using an SPI interface tocommunicate a clock signal and a first message from the first chiplet,or primary device, to a second chiplet, or secondary device. In anexample, block 604 can include using the primary device 312 to providethe clock signal 320 to the SPI memory device 302 and to concurrentlyprovide the first message, such as comprising a portion of the secondarydevice input signal 322, to the SPI memory device 302. For example,block 604 can include a portion of a write instruction, such asillustrated in FIG. 5, and can optionally include sending a message fromthe primary device 312 to the SPI memory device 302, where the messageincludes the secondary device identification field 510 and the datapayload 512.

Following block 604, the first chiplet completes its outgoing messageand, at block 606, the first chiplet can be configured to listen to ormonitor a data input port for a response message. For example, theprimary device 312 can be configured to monitor the data input port 338for a response message from the second chiplet.

At block 608, the primary device 312 can be configured to useinformation from the data input port 338 to identify a secondary devicestatus reply. In an example, the secondary device status reply caninclude information that is received or sensed at a data input port ofthe first chiplet, and can optionally include a message prepared by thesecond chiplet and sent to the first chiplet.

FIG. 7 illustrates generally a flow diagram of an example of a secondmethod 700 that can include analyzing a secondary device status replyreceived by a first chiplet. In an example, the second method 700 beginsat block 702 with identifying or receiving a secondary device statusreply at a primary device or first chiplet. In an example, block 702 caninclude or correspond to block 608 from the example of FIG. 6.

Following block 702, the second method 700 can continue at decisionblock 704. At decision block 704, the first chiplet can determinewhether the secondary device status reply corresponds to a pull-upsignal that is present at a data input port of the first chiplet.Characteristics of the pull-up signal, such as signal magnitudeinformation, can be known a priori by the first chiplet.

If, at decision block 704, the secondary device status reply does notcorrespond to the pull-up signal, then the second method 700 proceeds atblock 706. For example, if a magnitude characteristic of the pull-upsignal is dissimilar from magnitude information in the secondary devicestatus reply, then the second method 700 can proceed to block 706. At706, the second method 700 can include using information in thesecondary device status reply to identify a secondary device statusfield of a response message. Information in the secondary device statusfield can indicate a status of a transaction between the first chipletor primary device and the second chiplet or secondary device. Forexample, the status can include, among other things, a communicationdeferral status, or a parity error (see, e.g., Table 2).

If, at decision block 704, the secondary device status reply doescorrespond to the pull-up signal, then the second method 700 proceeds atblock 708. For example, if a magnitude characteristic of the pull-upsignal is similar to magnitude information in the secondary devicestatus reply, then the second method 700 can proceed to block 708. In anexample, the secondary device status reply can include informationreceived or sensed at a data input port of the first chiplet about apull-up signal or buffer reference value at the port. In other words,the secondary device status reply can include information sensed at theinput port of the first chiplet when no other endpoint or secondarydevice or chiplet actively sends a response message to the firstchiplet. In an example when the secondary device status reply includesthe pull-up signal or reference value information, such as over adesignated number of clock cycles, then the first chiplet or primarydevice 312 can determine the second chiplet, or SPI memory device 302,was not successfully enabled, was not successfully reached by the firstmessage, or that the second chiplet is not configured to understand orrespond to the first message.

FIG. 8 illustrates a block diagram of an example machine 800 with which,in which, or by which any one or more of the techniques (e.g.,methodologies) discussed herein can be implemented. Examples, asdescribed herein, can include, or can operate by, logic or a number ofcomponents, or mechanisms in the machine 800. Circuitry (e.g.,processing circuitry) is a collection of circuits implemented intangible entities of the machine 800 that include hardware (e.g., simplecircuits, gates, logic, etc.). Circuitry membership can be flexible overtime. Circuitries include members that can, alone or in combination,perform specified operations when operating. In an example, hardware ofthe circuitry can be immutably designed to carry out a specificoperation (e.g., hardwired). In an example, the hardware of thecircuitry can include variably connected physical components (e.g.,execution units, transistors, simple circuits, etc.) including a machinereadable medium physically modified (e.g., magnetically, electrically,moveable placement of invariant massed particles, etc.) to encodeinstructions of the specific operation. In connecting the physicalcomponents, the underlying electrical properties of a hardwareconstituent are changed, for example, from an insulator to a conductoror vice versa. The instructions enable embedded hardware (e.g., theexecution units or a loading mechanism) to create members of thecircuitry in hardware via the variable connections to carry out portionsof the specific operation when in operation. Accordingly, in an example,the machine readable medium elements are part of the circuitry or arecommunicatively coupled to the other components of the circuitry whenthe device is operating. In an example, any of the physical componentscan be used in more than one member of more than one circuitry. Forexample, under operation, execution units can be used in a first circuitof a first circuitry at one point in time and reused by a second circuitin the first circuitry, or by a third circuit in a second circuitry at adifferent time. Additional examples of these components with respect tothe machine 800 follow.

In alternative embodiments, the machine 800 can operate as a standalonedevice or can be connected (e.g., networked) to other machines. In anetworked deployment, the machine 800 can operate in the capacity of aserver machine, a client machine, or both in server-client networkenvironments. In an example, the machine 800 can act as a peer machinein peer-to-peer (P2P) (or other distributed) network environment. Themachine 800 can be a personal computer (PC), a tablet PC, a set-top box(STB), a personal digital assistant (PDA), a mobile telephone, a webappliance, a network router, switch or bridge, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein, such as cloud computing, software as aservice (SaaS), other computer cluster configurations.

The machine 800 (e.g., computer system) can include a hardware processor802 (e.g., a central processing unit (CPU), a graphics processing unit(GPU), a hardware processor core, or any combination thereof), a mainmemory 808, a static memory 810 (e.g., memory or storage for firmware,microcode, a basic-input-output (BIOS), unified extensible firmwareinterface (UEFI), etc.), and mass storage 812 (e.g., hard drives, tapedrives, flash storage, or other block devices) some or all of which cancommunicate with each other via an interlink 818 (e.g., a bus, such asan SPI bus). The machine 800 can further include a display device 820,an alphanumeric input device 822 (e.g., a keyboard), and a userinterface (UI) navigation device 824 (e.g., a mouse). In an example, thedisplay device 820, input device 822, and navigation device 824 can be atouch screen display. The machine 800 can additionally include a massstorage 812 (e.g., drive unit), a signal generation device 828 (e.g., aspeaker), a network interface device 814, and one or more sensor(s) 826,such as a global positioning system (GPS) sensor, compass,accelerometer, or other sensor. The machine 800 can include an outputcontroller 830, such as a serial (e.g., universal serial bus (USB),parallel, or other wired or wireless (e.g., infrared (IR), near fieldcommunication (NFC), etc.) connection to communicate or control one ormore peripheral devices (e.g., a printer, card reader, etc.).

Registers of the processor 802, the main memory 808, the static memory810, or the mass storage 812 can be, or include, a machine-readablemedium 806 on which is stored one or more sets of data structures orinstructions 804 (e.g., software) embodying or utilized by any one ormore of the techniques or functions described herein. The instructions804 can also reside, completely or at least partially, within any ofregisters of the processor 802, the main memory 808, the static memory810, or the mass storage 812 during execution thereof by the machine800. In an example, one or any combination of the hardware processor802, the main memory 808, the static memory 810, or the mass storage 812can constitute the machine-readable medium 806 or media. While themachine-readable medium 806 is illustrated as a single medium, the term“machine readable medium” can include a single medium or multiple media(e.g., a centralized or distributed database, or associated caches andservers) configured to store the one or more instructions 804. In anexample, the various memory units or processor 802 can becommunicatively coupled using a bus such as an SPI bus.

The term “machine readable medium” can include any medium that iscapable of storing, encoding, or carrying instructions for execution bythe machine 800 and that cause the machine 800 to perform any one ormore of the techniques of the present disclosure, or that is capable ofstoring, encoding or carrying data structures used by or associated withsuch instructions. Non-limiting machine readable medium examples caninclude solid-state memories, optical media, magnetic media, and signals(e.g., radio frequency signals, other photon based signals, soundsignals, etc.). In an example, a non-transitory machine readable mediumcomprises a machine readable medium with a plurality of particles havinginvariant (e.g., rest) mass, and thus are compositions of matter.Accordingly, non-transitory machine-readable media are machine readablemedia that do not include transitory propagating signals. Specificexamples of non-transitory machine readable media can include:non-volatile memory, such as semiconductor memory devices (e.g.,electrically programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM)) and flash memorydevices; magnetic disks, such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

In an example, information stored or otherwise provided on themachine-readable medium 806 can be representative of the instructions804, such as instructions 804 themselves or a format from which theinstructions 804 can be derived. This format from which the instructions804 can be derived can include source code, encoded instructions (e.g.,in compressed or encrypted form), packaged instructions (e.g., splitinto multiple packages), or the like. The information representative ofthe instructions 804 in the machine-readable medium 806 can be processedby processing circuitry into the instructions to implement any of theoperations discussed herein. For example, deriving the instructions 804from the information (e.g., processing by the processing circuitry) caninclude: compiling (e.g., from source code, object code, etc.),interpreting, loading, organizing (e.g., dynamically or staticallylinking), encoding, decoding, encrypting, unencrypting, packaging,unpackaging, or otherwise manipulating the information into theinstructions 804.

In an example, the derivation of the instructions 804 can includeassembly, compilation, or interpretation of the information (e.g., bythe processing circuitry) to create the instructions 804 from someintermediate or preprocessed format provided by the machine-readablemedium 806. The information, when provided in multiple parts, can becombined, unpacked, and modified to create the instructions 804. Forexample, the information can be in multiple compressed source codepackages (or object code, or binary executable code, etc.) on one orseveral remote servers. The source code packages can be encrypted whenin transit over a network and decrypted, uncompressed, assembled (e.g.,linked) if necessary, and compiled or interpreted (e.g., into a library,stand-alone executable etc.) at a local machine, and executed by thelocal machine.

The instructions 804 can be further transmitted or received over acommunication network 816 using a transmission medium via the networkinterface device 814 utilizing any one of a number of transfer protocols(e.g., frame relay, internet protocol (IP), transmission controlprotocol (TCP), user datagram protocol (UDP), hypertext transferprotocol (HTTP), etc.). Example communication networks can include alocal area network (LAN), a wide area network (WAN), a packet datanetwork (e.g., the Internet), mobile telephone networks (e.g., cellularnetworks), plain old telephone (POTS) networks, and wireless datanetworks (e.g., institute of Electrical and Electronics Engineers (IEEE)802.11 family of standards known as Wi-Fi®, IEEE 802.16 family ofstandards known as WiMax@), IEEE 802.15.4 family of standards,peer-to-peer (P2P) networks, among others. In an example, the networkinterface device 814 can include one or more physical jacks (e.g.,Ethernet, coaxial, or phone jacks) or one or more antennas to connect tothe communication network 816. In an example, the network interfacedevice 814 can include a plurality of antennas to wirelessly communicateusing at least one of single-input multiple-output (SIMO),multiple-input multiple-output (MIMO), or multiple-input single-output(MISO) techniques. The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding orcarrying instructions for execution by the machine 800, and includesdigital or analog communications signals or other intangible medium tofacilitate communication of such software. A transmission medium is amachine readable medium.

The following are examples or devices and methods in accordance with theteachings herein. In an example, one or more of the following examplescan be used with an SPI system to confirm a presence or absence of asecondary device in the system and/or to carry out deferred transactionsbetween primary and second devices or chiplets in the system.

Example 1 can include an apparatus that includes a primary chipletcouplable to multiple secondary chiplets using a serial peripheralinterface (SPI) interface configured for an SPI protocol, wherein theprimary chiplet includes first and second data ports coupled torespective first and second data channels of the SPI interface, whereinthe primary chiplet is configured to send a first message using thefirst data port, the first message comprising a command field configuredto access a first selected chiplet of the multiple secondary chiplets,the first selected chiplet configured as an SPI secondary device. InExample 1, the primary chiplet can be configured to receive, using thesecond data port, information about a communication status of themultiple secondary chiplets, including information about whether thefirst message successfully reached the first selected chiplet.

Example 2 can include or use the features of Example 1, wherein theprimary chiplet is configured to receive a response message from thefirst selected chiplet in response to the first message, and theresponse message includes a secondary device status field that indicatesthe first message successfully reached the first selected chiplet.

Example 3 can include or use the features of Example 2, wherein thesecondary device status field indicates a readiness of the firstselected chiplet to provide a particular data payload to the primarychiplet.

Example 4 can include or use the features of Example 2 or Example 3,wherein the secondary device status field includes one or more bits setto indicate a deferred return status. In Example 4, the primary chipletcan be further configured to, in response to receiving a responsemessage with one or more bits set to indicate the deferred returnstatus, send a deferred read request to the first selected chiplet.

Example 5 can include or use the features of Examples 2-4, wherein thesecondary device status field can include one or more bits set toindicate a data parity error.

Example 6 can include or use features of any of the preceding examples,wherein the primary chiplet includes circuitry configured to provide apull-up signal at the second data port.

Example 7 can include or use the features of Example 6, wherein theprimary chiplet is configured to identify the information about thecommunication status of the multiple secondary chiplets using thepull-up signal.

Example 8 can include or use the features of Examples 6 or 7, wherein inthe absence of a response message on the second data channel responsiveto the first message, the primary chiplet can be configured to identifythe information about the communication status of the multiple secondarychiplets using the pull-up signal.

Example 9 can include or use features of any of the preceding examples,wherein the information about the communication status of the multiplesecondary chiplets indicates an absence of the multiple secondarychiplets from the second data channel.

Example 10 can include or use features of any of the preceding examples,wherein Example 10 can further comprise the multiple secondary chiplets,wherein each of the multiple secondary chiplets includes a respectivedata output port coupled to the second data channel of the SPIinterface, and each of the multiple secondary chiplets can be configuredto tri-state its respective data output when it is unused forcommunication with the primary chiplet.

Example 11 can include a system comprising a primary chiplet coupled tomultiple secondary chiplets using a serial peripheral interface (SPI)interface configured to use an SPI protocol, and a first selectedchiplet of the multiple secondary chiplets, the first selected chipletconfigured to interface with the primary chiplet using the SPIinterface. In Example 11, the primary chiplet can be configured toperform operations comprising, e.g., send a clock signal to the firstselected chiplet using a clock channel of the SPI interface, send afirst message using a first data channel of the SPI interface, the firstmessage comprising a command field and a register address field, thecommand field including one or more bits to enable a controller on thefirst selected chiplet, and the register address field including one ormore bits to address a particular memory register on the first selectedchiplet, and receive, using a second data channel of the SPI interface,communication status information about the multiple secondary chiplets.

Example 12 can include or use the features of Example 11, wherein theprimary chiplet can be further configured to perform operationscomprising, e.g., send a chip select signal to the first selectedchiplet using a chip select channel of the SPI interface, and receive aresponse message from the first selected chiplet using the second datachannel of the SPI interface, wherein the response message includes asecondary device status field that indicates a status of a transactionbetween the primary chiplet and the first selected chiplet.

Example 13 can include or use the features of Example 12, wherein thesecondary device status field includes one or more bits set to indicatea deferred return status, and the primary chiplet can be furtherconfigured to, in response to receiving the response message, send adeferred read request to the first selected chiplet.

Example 14 can include or use features of any of Examples 11-13, whereinthe primary chiplet can include a data input port coupled to the seconddata channel of the SPI interface, and the primary chiplet can beconfigured to provide a pull-up signal at the data input port.

Example 15 can include or use features of any of Examples 11-14, whereinthe primary chiplet is configured to determine a communication status ofthe multiple secondary chiplets based on a magnitude characteristic ofthe pull-up signal.

Example 16 can include or use features of any of Examples 14 or 15,wherein when the primary chiplet detects the pull-up signal at the datainput port over a series of pulses of the clock signal, the primarychiplet is configured to determine the multiple secondary chiplets,including the first selected chiplet, are decoupled from the second datachannel of the SPI interface.

Example 17 can include a method for determining whether a secondarydevice is available to communicate with a primary device using a serialperipheral interface (SPI) interface configured for an SPI protocol,wherein the primary device includes a data input port and a data outputport, and wherein the primary device is configured to provide a pull-upsignal at the data input port. Example 17 can include steps comprising,at the primary device, sending a clock signal using a clock port and aclock channel of the SPI interface, sending a first message using thedata output port and a first data channel of the SPI interface, thefirst message comprising a command field configured to access thesecondary device, receiving information from the data input port, anddetermining a presence or absence of the secondary device on the SPIinterface based on a relationship between a reference magnitude of thepull-up signal and the information received from the data input port.

Example 18 can include or use the features of Example 17, whereindetermining the presence or absence of the secondary device on the SPIinterface can include using information about the reference magnitude ofthe pull-up signal and using information received from the data inputport over multiple sequential cycles of the clock signal.

Example 19 can include or use features of any of Examples 17 or 18, andcan further include, at the secondary device, receiving the firstmessage and, in response, sending a response message to the primarydevice using a second data channel of the SPI interface, wherein thedata input port of the primary device is coupled to the second datachannel, and wherein the response message includes a secondary devicestatus field that indicates a readiness of the secondary device toprovide a particular data payload to the primary device.

Example 20 can include or use features of any of Examples 17-19, and canfurther include, at the primary device, receiving a response messagefrom the secondary device in response to the first message, identifyinga secondary device status field of the response message, and determiningwhether the secondary device status field indicates a deferral, acomplete transaction, or a communication error.

Example 21 can include or use features of any of Examples 17-20, and canfurther include, at the secondary device, maintaining a secondary devicedata output port in a tri-stated state, the secondary device data outputport coupled to the data input port of the primary device via a seconddata channel of the SPI interface, receiving the first message,determining whether the first message includes a device identifiercorresponding to the secondary device, and when the first messageincludes the device identifier corresponding to the secondary device,providing a data signal at the secondary device data output port,wherein the data signal has signal values that are other than a value ofthe tri-stated state.

Each of the above Examples can be combined or used together in variousways to carry out communications over a synchronous interface, such asover an SPI interface.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments in which theinvention can be practiced. These embodiments are also referred toherein as “examples”. Such examples can include elements in addition tothose shown or described. However, the present inventors alsocontemplate examples in which only those elements shown or described areprovided. Moreover, the present inventors also contemplate examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” can include “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein”. Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to impose numerical requirements on their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) can be used in combination with each other. Otherembodiments can be used, such as by one of ordinary skill in the artupon reviewing the above description. It is submitted with theunderstanding that it will not be used to interpret or limit the scopeor meaning of the claims. Also, in the above Detailed Description,various features can be grouped together to streamline the disclosure.This should not be interpreted as intending that an unclaimed disclosedfeature is essential to any claim. Rather, inventive subject matter canlie in less than all features of a particular disclosed embodiment.Thus, the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separateembodiment, and it is contemplated that such embodiments can be combinedwith each other in various combinations or permutations. The scope ofthe invention should be determined with reference to the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

What is claimed is:
 1. An apparatus comprising: a primary chipletcouplable to multiple secondary chiplets using a serial peripheralinterface (SPI) interface configured for an SPI protocol, wherein theprimary chiplet includes first and second data ports coupled torespective first and second data channels of the SPI interface, whereinthe primary chiplet is configured to: send a first message using thefirst data port, the first message comprising a command field and achiplet identification field, the command field configured to access afirst selected chiplet of the multiple secondary chiplets, the firstselected chiplet having an identifier corresponding to the chipletidentification field and the first selected chiplet configured as an SPIsecondary device; and receive, using the second data port, a responsemessage from the first selected chiplet in response to the firstmessage, wherein the response message comprises a secondary devicestatus field that includes information from the first selected chipletabout whether a register operation successfully completed, whether aregister protocol or parity error exists, and whether a registeroperation was deferred.
 2. The apparatus of claim 1, wherein thesecondary device status field indicates a readiness of the firstselected chiplet to provide a particular data payload to the primarychiplet.
 3. The apparatus of claim 1, wherein the secondary devicestatus field includes one or more bits set to indicate a deferred returnstatus; and wherein the primary chiplet is further configured to, inresponse to receiving the response message with one or more bits set toindicate the deferred return status, send a deferred read request to thefirst selected chiplet.
 4. The apparatus of claim 1, wherein the primarychiplet includes circuitry configured to provide a pull-up signal at thesecond data port.
 5. The apparatus of claim 4, wherein the primarychiplet, is configured to use the pull-up signal to identify,concurrently, information about a communication status of all of themultiple secondary chiplets.
 6. The apparatus of claim 2, wherein theprimary chiplet is configured to identify an absence of all of themultiple secondary chiplets using the pull-up signal.
 7. The apparatusof claim 1, further comprising the multiple secondary chiplets, whereineach of the multiple secondary chiplets includes a respective dataoutput port coupled to the second data channel of the SPI interface, andwherein each of the multiple secondary chiplets is configured totri-state its respective data output when it is unused for communicationwith the primary chiplet.
 8. A system comprising: a primary chipletcoupled to multiple secondary chiplets using a serial peripheralinterface (SPI) interface configured to use an SPI protocol; and a firstselected chiplet of the multiple secondary chiplets, the first selectedchiplet configured to interface with the primary chiplet using the SPIinterface; wherein the primary chiplet is configured to performoperations comprising: send a clock signal to the first selected chipletusing a clock channel of the SPI interface; send a first message using afirst data channel of the SPI interface, the first message comprising acommand field, a chiplet identification field, and a register addressfield, the command field including one or more bits to enable acontroller on the first selected chiplet, and the register address fieldincluding one or more bits to address a particular memory register onthe first selected chiplet, wherein the first selected chiplet isidentified by information in the chiplet identification field; andreceive, using a second data channel of the SPI interface, a response tothe first message from the first selected chiplet, the responsecomprising a device status field, wherein the device status fieldincludes information about whether a register operation successfullycompleted at the first selected chiplet, whether a register protocol orparity error exists at the first selected chiplet, and whether aregister operation was deferred at the first selected chiplet.
 9. Thesystem of claim 8, wherein e primary chiplet is further configured toperform operations comprising: send a chip select signal to the firstselected chiplet using a chip select channel the SPI interface; whereinthe device status field indicates a status of a transaction between theprimary chiplet and the first selected chiplet.
 10. The system of claim9, wherein the device status field includes one or more bits set toindicate a deferred return status; and wherein the primary chiplet isfurther configured to, in response to receiving the response message,send a deferred read request to the first selected chiplet.
 11. Thesystem of claim 8, wherein the primary chiplet comprises a data inputport coupled to the second data channel of the SPI interface, andwherein the primary chiplet is configured to provide a pull-up signal atthe data input port.
 12. The system of claim 11, wherein the primarychiplet is configured to determine a communication status of all of themultiple secondary chiplets based on a magnitude characteristic of thepull-up signal at the same data input port.
 13. The system of claim 11,wherein when the primary chiplet detects the pull-up signal at the datainput port over a series of pulses of the clock signal, the primarychiplet is configured to determine the multiple secondary chiplets,including the first selected chiplet, are decoupled from the second datachannel of the SPI interface.
 14. A method for determining whether asecondary device is available to communicate with a primary device usinga serial peripheral interface (SPI) interface configured for an SPIprotocol, wherein the primary device includes a data input port and adata output port, and wherein the primary device is configured toprovide a pull-up signal at the data input port, the method comprising:at the primary device, sending a clock signal using a clock port and aclock channel of the SPI interface; sending a first message using thedata output port and a first data channel of the SPI interface, thefirst message comprising a command field configured to access thesecondary device, a device identification field configured to identifythe secondary device from other devices coupled to the first datachannel of the SPI interface, a register address field configured toidentify a particular register location at the secondary device, and adata message field; receiving signal magnitude information from the datainput port over multiple sequential cycles of the clock signal; anddetermining an operation status of the secondary device or absence ofthe secondary device based on a relationship between a referencemagnitude of the pull-up signal and the signal magnitude informationreceived from the data input port over the multiple sequential cycles ofthe clock signal.
 15. The method of claim 14, further comprising: at thesecondary device, receiving the first, message and, in response, sendinga response message to the primary device using a second data channel ofthe SPI interface, wherein the data input port of the primary device iscoupled to the second data channel, and wherein the response messageincludes a multiple-bit secondary device status field that indicates aregister operation status of the secondary device alone or indicates aregister operation status of the secondary device together with a datapayload; and wherein determining the operation status of the secondarydevice includes using the multiple-bit secondary device status field.16. The method of claim 14, further comprising: at the primary device:receiving a response message from the secondary device in response tothe first message; identifying a multiple-hit secondary device statusfield of the response message; and determining whether the secondarydevice status field indicates a deferral, determining whether thesecondary device status field indicates a complete transaction, anddetermining whether the secondary device status field indicates acommunication error.
 17. The method of claim 14, further comprising: atthe secondary device, maintaining a secondary device data output port ina tri-stated state, the secondary device data output port coupled to thedata input port of the primary device via a second data channel of theSPI interface; receiving the first message; determining whether thefirst message includes a device identifier corresponding to thesecondary device; and when the first message includes the deviceidentifier corresponding to the secondary device, providing a datasignal at the secondary device data output port, wherein the data signalhas signal values that are other than a value of the tri-stated state.18. The method of claim 14, wherein information in the command field ofthe first message indicates whether the first message includes adeferred read request.